Data Services
Overview
Data Services specify how to reach a data endpoint and which Qarbine compute node performs the interaction. They are listed in the tab shown below.
Components reference data services by (case sensitive) name which allows components in Qarbine installation X (i.e., test) to use the same data service names as Y (i.e., production) but point to completely different data servers.
A Data Service defines how to interact with a source of data. That source may range from a traditional database, to a RESTful endpoint. You can control which data services are visible to a principal or access group. A variety of databases are supported including traditional SQL databases along with modern NoSQL databases. Their availability varies by Qarbine edition and license level.
Qarbine clients connect to a Qarbine compute node specified by the Compute URL. The data service there uses the Data Service's Server Template to interact with the data endpoint. The chosen driver knows how to perform the API interactions with the endpoint. The supplied parameters may contain environment variables which are resolved relative to the compute node. A depiction is shown below.
Client ↔ Compute Node with Data Service(s) ↔ Data Endpoint(s)
A sample Data Endpoint would be MongoDB Atlas. A compute node may contain several data services to access multiple data endpoints. You can also deploy multiple Qarbine compute nodes to place them closer to their target data servers. This can improve performance, help adherence to regulations such as GDPR, and avoid egress charges.
The Data Service name provides a level of indirection to your queries. A testing environment may configure 'Sales' to reference a test database. You can then develop Qarbine components which reference the 'Sales' data service. Once completed, you can export the component and then import it into your production environment. That Qarbine installation also has a 'Sales' data service defined, but its details reference the production data location.
Qarbine uses several internal database accounts in order to access its internal database content. These accounts are different from those used to access your own data that you wish to retrieve and analyze. The latter are defined as Qarbine Data Services and you control their settings which should always be read-only in nature.
Built-in Data Services
Each Data Service entry specifies what driver to use to access the data, the data endpoint, and other details. For example, the “Qarbine catalog” data service uses the MongoDB driver to access the internal Qarbine catalog running on the primary Qarbine host. Several data services are automatically defined during system startup using the current configuration settings.
| Name | Auto | Database | Uses DB Account | Purpose |
|---|---|---|---|---|
| Qarbine configuration | Y | Qarbine | q_admin | Full reporting including configuration |
| Qarbine catalog | Y | Qarbine | q_user | Catalog browser relationship reports |
| Qarbine log | Y | Log | q_log_report | Event log reporting |
| Sample Data Service | N | Public MongoDB Atlas | Readily available sample data. | |
| Your data service1..n | Your database | Per your server template and other options | Access your data |
Adding a Data Service
A Data Service is created by clicking
The middle area shows the following.
Specify a meaningful name. The name is an alias used by various tools and data access settings. Components such as a Data Source reference the data service by name. Providing a description is a good way to add notes about the data service.
Next indicate the default compute node to access a particular data server.
Next indicate the data driver to use for this data service.
The settings for each driver vary a great deal. See the Data Service Configuration folder document for information on the various drivers and their options.
Using Environment Variables
Some drivers may also use environment variables to control their operation. These environment variables can be set in the standard manner for Ubuntu and Amazon Linux.
Managing Who Can Use A Data Service
After setting up the driver as appropriate, indicate for which users this data service is enabled.
